

United States Patent and Trademark Office 




UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



| APPLICATION NO. | 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/630,061 


07/30/2003 


Joel M. Barris 


012-001 


8188 



53961 7590 

FALKOWSKI PLLC 
P.O. BOX 650 
NOVI, MI 48376-0650 



04/05/2007 



EXAMINER 



BAUTISTA, XIOMARA L 



ART UNIT 



PAPER NUMBER 



2179 



SHORTENED STATUTORY PERIOD OF RESPONSE 



MAIL DATE 



DELIVERY MODE 



3 MONTHS 



04/05/2007 



PAPER 



Please find below and/or attached an Office communication concerning this application or proceeding. 



If NO period for reply is specified above, the maximum statutory period will apply and will expire 6 MONTHS 
from the mailing date of this communication. 



PTOL-90A (Rev. 10/06) 



Office Action Summarv 

^mw m u m \* \*r M 1 VP mm \mf>m m *mmt \J§ 1 1 III 1 V4 V V 


Application No. 

10/630,061 


Applicant(s) 

JOEL M. BARRIS ET AL 


Examiner 

X. L. Bautista 


Art Unit 

2179 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address -- 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 



1 Responsive to communication(s) filed on 30 July 2003 . 
2a)Q This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) £3 Claim(s) 7-36 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) £3 Claim(s) 35 is/are allowed. 



6) |El Claim(s). 1-29,31,33.34 and 36 is/are rejected. 

7) K Claim(s) 30 and 32 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

§)□ The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 30 July 2003 is/are: a)[X] accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

1 2)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.0 Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) K Notice of References Cited (PTO-892) 

2) [J Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO/SB/08) 

Paper No(s)/Mail Date 4/8/04 . 



4) O Interview Summary (PTO-413) 

Paper No{s)/Mail Date. . 

5) L Notice of Informal Patent Application 

6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 20070322 



Application/Control Number: 10/630,061 Page 2 

Art Unit: 2179 

DETAILED ACTION 

Claim Rejections - 35 USC § 102 

1. The following is a quotation of the appropriate paragraphs of 35.U.S.C. 102 
that form the basis for the rejections under this section made in this Office action' 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in 
the United States before the invention thereof by the applicant for patent, or on an international 
application by another who has fulfilled the requirements of paragraphs (l), (2), and (4) of section 
371(c) of this title before the invention thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection 
Act of 1999 (AIPA) and the Intellectual Property and High Technology Technical 
Amendments Act of 2002 do not apply when the reference is a U.S. patent resulting 
directly or indirectly from an international application filed before November 29, 
2000. Therefore, the prior art date of the reference is determined under 35 U.S.C. 
102(e) prior to the amendment by the AIPA (preAJPA 35 U.S.C. 102(e)). 

2. Claims 1-9, 11-15, 17, 22-25, 27, 28, 31, 33, 34 and 36 are rejected under 35 
U.S.C. 102(e) as being anticipated by Pea etal (US 7,082,572 B2). 

Claims 1 and 36- 

Pea discloses a method and interactive system for authoring and analyzing 
digital video content (abstract; col. 1, lines 54*67; col. 2, lines 6-20). Pea teaches 
representations of physical spaces (col. 3, lines 31-57). Pea teaches a plurality of 
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viewpoints including a first video clip (col. 4, lines 48*67; col. 5, lines 1*14). Pea 
teaches interactive navigation of panoramic digital imagery (col. 5, lines 15 _ 67; col. 
.6, lines 1-8; col. 19, lines 30 _ 50). Pea teaches transitions from one scene to another 
(abstract; col. 19, lines 30-50). 
Claim 2: 

Pea teaches a terminal connected to a computer network (fig. 11). 
Claim 3: 

Pea teaches a computer network being the Internet (col. 9, lines 13*33). 
Claim 4: 

See claim 1. Pea teaches a plurality of viewpoints including video clips (figs. 
2A-4, 7-9). 
Claim 5: 

Pea teaches the system allows motion in different directions (col. 5, lines 31- 
67; col. 6, lines 1-8). 
Claim 6: 

Pea teaches video clips can be created form still images motion video, etc. 
(col. 1, lines 54*67; col. 2, lines 1-5; col. 4, lines 40*45). 
Claim 7: 

Pea teaches that video clips may be captured with a standard video camera 
from the actual physical space portrayed by the representation (col. 4, lines 48-57; 
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col. 5, lines 15-30; col. 10, lines 5-20). 
Claim 8: 

Pea teaches data markers corresponding to traversal records plotted against 
an interactive abstract map to enable users to shift between levels of abstraction in 
exploring the video record (abstract; p. 2, par. 0013; p. 12, par. 0135-0136). 
Claim 9: 

Pea teaches a zoom tool for zooming in and zooming out of a view (abstract; p. 
1, par. 0008, 0010; p. 3, par. 0031; p. 6, par. 0063). 
Claim 11: 

* 

Pea teaches a layout (blueprint) of the representation (col. 2, lines 6-13; col. 5, 
lines 31-42; figs. 7-8). 
Claim 12: 

Pea teaches interactive, electronic exploration and analysis of visual data 
using a plurality of traversal records; displaying an abstract map; plotting a 
plurality of markers (viewpoint object), each corresponding to a traversal record in a 
plurality of locations on the map; and playing back the traversal in response to user 
selection of the marker (col. 3, lines 35 _ 60). 
Claim 13: 

Pea teaches an object including at least one of an object video clip or an object 
still image (col. 1, lines 54-67; col. 6, lines 35-45); an object location within the 
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representation (col. 3, lines 17*60! col. 5, lines 58-67; col. 6, lines 1-8, 55-66; col. 7, 
lines 30-38); and a navigation tool for activating an object video clip (fig. 7; col. 19, 
lines 6- 50) . 
Claim 14: 

See claims 11 and 13. Pea teaches blueprint and objects identifying a 
location (fig. 7; col. 19, lines 6*50). 
Claim 15: 

Pea teaches a pan tool wherein a second view is a pan view of a first view (col. 
2, lines 6*13, 49-61; col. 5, lines 15-31; col. 10, lines 12-20). 
Claim 17: 

Pea teaches a slider bar for selecting and displaying traversal records and for 
other types of selections (col. 3, lines 53*60; col. 7, lines 8-14; col. 19, lines 30-49). 
Claim 22: 

Pea teaches "point-of-view" traversals and a plurality of objects (col. 1, lines 
26-30; col. 4, lines 48-52; col. 5, lines 26-30; figs. 2A-4 and 7-9). 
Claim 23: 

Pea teaches classification codes, which are predefined tags selected from a 
coding template menu; this feature allows users to classify traversals wherein the 
classification labels encoding template are user definable (col. 2, lines 1-5; col. 6, 
lines 16-26; fig. 4). 
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Claim 24: 

Pea teaches invocation of information and selectively accessing audio 
(narrative) depending on the current view (col. 2, lines 1"5, 40-47, 62-66! col. 3, lines 
1- 11; col. 7, lines 1-14). 
Claim 25: 

Pea teaches a worksheet region list of annotated traversal records in a web 
document accessible via network using a browser (abstract; col. 2, lines 45*48; col. 3, 
lines 15-32) and a graphical user interface that enables navigating and interacting 
with a display for viewing a representation (col. 5, lines 31-57; col. 6, lines 9-15; figs. 
2A-4, 7 and 9). 
Claim 27: 

Pea teaches a browser that may or may not be supported by a high-speed 
connection (col. 9, lines 34-60; col. 12, lines 23*38). 
Claim 28: 

Pea teaches a video clip that does not include a distorted image (col. 5, lines 

* 

31-57-49). 
Claim 31: 

Pea teaches that the use can click on a button (with a click tool) on the screen 
video control region (active clickable area) and that will initiate an action (invoke a 
scripted or programmed behavior), (col. 6, lines 62-66). 
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Claims 33 and 34: 

Pea teaches supplementary information (ancillary content) that may be 
invoked based on a current view. Pea teaches that the visual data may have a 
spatial subset of the visual data, and storing a record of a traversal may include 
spatial audio data associated with the visual data in each frame. The visual data 
may depict a concert hall or conference room, and the spatial audio may comprise a 
recording of music in the concert hall or conversation in the conference room. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action- 

* 

(a) A patent may not be obtained though the invention is not identically disclosed or described as 
set forth in section 102 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious at 
the time the invention was made to a person having ordinary skill in the art to which said subject 
matter pertains. Patentability shall not be negatived by the manner in which the invention was 
made. 

4. Claims 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Xiang ZHANG et al (article entitled Taking AR into Large Scale Industrial 
Environment- Navigation and Information Access with Mobile Computers , 
published 2001). 

* 

Claim 10: 

Pea teaches navigation and traversal of visual data but it does not teach a 
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compass for indicating the direction of a current view of a representation. However, 
Zhang discloses a framework designed for spatial data access, on-site navigation 
and real-time video augmentation. Zhang teaches a computer having a camera for 
observing the environment for visual coded markers; the markers being registered 
to a global coordinate system through drawings or floor plans . The invention 
processes images coming from the camera and estimates the location of the user. 
The system can guide the user through the environment, provide location relevant 
data from a spatial database, and augment the view of the user through the camera 
(abstract; col. 1, page l). Zhang teaches an interface for displaying the video images 
and a navigation guidance arrow (compass), (fig. 2; col. 1, page 2). Therefore, it 
would have been obvious to one having ordinary skill in the art at the time the 
invention was made to modify Pea's method of displaying panoramic images to 
include Zhang's teaching of displaying objects for guiding the users because they 
provide the users with a visual indicator or marker for informing about the user's 
position if the user unsure where he is or disoriented. 

« 

5. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Chen et a/ (US 6,268,864 Bl). 

Claim 16: 

Pea teaches navigation of panoramic images but it does not teach a tilt tool. 



Application/Control Number: 10/630,061 Page 9 

Art Unit: 2179 

However, Chen discloses a user interface having navigation tools for enabling a user 
to tilt, pan and zoom a perspective view of a panorama (coL 23, lines 61*67; col. 24, 
lines 1-23). Thus, it would have been obvious to a persona having ordinary skill in 
the art at the time of invention to modify Pea's method of navigating panoramic 
images to include Chens teaching of a user interface having icons for tilting, 
zooming or panning images because the user is able to view images from different 
sides, angles, and distances, whenever a portion of the image calls his attention. 

» 

6. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Cohen et al (US 6,040,841). 

Claim 18: 

Pea does not teach that video clips are shot using shoot heuristics. However, 
Cohen discloses a method for generating specifications for displaying events in 

■ 

computer animated environments. Cohen teaches heuristics are used for selecting 
good shots in order to create good scenes (col. 7, lines 12-47; coL 22, lines 49-50). 
Therefore, it would have been obvious to one ordinarily skilled in the art at the time 
the invention was made to modify Pea's method for interactive authoring of digital 
video content to include Cohen's teaching of using shoot heuristics because rules can 
be used to influence the relevance measurements, for example, heuristics may be 
used to assign particular importance to multimedia scene shots containing 



Application/Control Number: 10/630,061 Page 10 

Art Unit: 2179 

superimposed text, or to multimedia scene shots preceding or following a camera 
pan/zoom can be used to bias the outcome of a multimedia summarization; or as 
Cohen says, heuristics are useful for selecting good shots. 

7. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Criminisi et al (US 6,987,520 B2). 

Claim 19: 

Pea does not teach video clips that are imported using import heuristics. 
However, Criminisi discloses a method and system for identifying appropriate 
material to replace destination regions in images. Criminisi explains that images 
are imported from other images using a set of rules (coL 7, lines 32-40). Thus, it 
would have been obvious to a person having ordinary skill in the art at the time the 

invention was made to use heuristics to import images because rules support 

■ 

performance of a plurality of functions, from provision of video clip overviews, 
through incorporation of special effects, and other techniques in order to import 
desired images and/or visual impacts into presentations. 

8. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Lin eta] (US 7, 120, 197 B2). 
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Claim 20: 

Pea does that each video clip is encoded using encode heuristics. However, 
Lin discloses a method and system for processing reference frames having a video 
encoder/decoder that filters reference frames to reduce discontinuities at block 
boundaries. Lin teaches an encoder/decoder that computes boundary heuristics for 
determining filtering, wherein the encoder computes measurements that depend on 
heuristics, which are used in the selection of reference frames and for improving the 
efficiency of motion estimation and compensationl (col. 17, lines 19*67). 
Thus, it would have been further obvious to a person having ordinary skill in the art 
at the time of invention to modify Pea's invention to include Lin's teaching of using 
heuristics to encode video clips because implementing rules may help improve the 
classification and selection of video clips and the altering of the video clip's 
characteristics. 

9. Claim 21 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Solomon (US 2003/00076722 Al). 

Claim 21 : 

Pea does not teach that each video clip is mapped to the representation in 
accordance with mapping heuristics. However, Solomon discloses a method for 
using a VLSI editor for displaying a VLSI design in photo -realistic detail in real" 
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* 

time. Solomon teaches that the invention uses different heuristics to determine 
which images will be mapped to the design (representation), (abstract; p. 5, par. 
0075, 0078, 0079; p. 8, par. 0108; p. 9, par. 0114). Thus, it would have been obvious 
to include Solomon's teaching of using heuristics to map images to a representation 
in Pea's invention because the system takes care of choosing proper images and the 
user does not have to worry about choosing images to generate a representation in 
image -realistic detail especially when the representation reaches into the tens or 
hundreds or thousands or millions of images. 

10. Claim 26 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Dymetman etaliUS 6,330,976 Bl). 

Claim 26: 

Pea does not teach a platform independent browser. However, Dymetman 
discloses a method and system for obtaining automatic actions through a network. 
Dymetman explains that it is sometimes difficult to obtain an appropriate 
automatic action such as access to multimedia information or other information 
available through a network especially when the context includes a physical object 
and the action should be appropriate to the object (abstract; col. 2, lines 48-65). 
Dymetman teaches an output player that allows an output to take the proper form . 
for each different computer platform. Dymetman explains that current browsers 
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provide these capabilities in a platform-independent way for computers (col. 24, 
lines 20-38). Therefore, it would have been obvious to one ordinarily skilled in the 
art at the time the invention was made to modify Pea's invention to include 
Dymetman's teaching of using platform-independent browser because the user is 
enabled to access all the features and functionalities of the invention or any 
application and will be provided with output for display by any Internet browser 

* 

and will be able to interact with it. 

11. Claim 29 is rejected under 35 U.S.C. 103(a) as being unpatentable over Pea 
and Seidman et a/ (US 7,167,840 Bl). 

Claim 29: 

Pea does not teach system interfaces with a consumer media player 
application. However, Seidman discloses a system and method for distributing and 
selling electronic content having a review device that is coupled to a consumer 
player application wherein the review device may be a web browser (col. 4, lines 50- 

■ 

62). Thus, it would have been obvious to a person having ordinary skill in the art at 
the time of invention to include Seidman's teaching in Pea's invention because the 
user is enabled to use the media player to play a presentation having video and/or 
audio when needing, for example, to play audio (such as music) provided by a third 
party. 
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Allowable Subject Matter 

* 

12. Claims 30 and 32 are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

13. Claim 35 is allowed. 

14. The following is a statement of reasons for the indication of allowable subject 
matter" Prior art of record fails to teach the combination of elements including a 
display subsystem having a plurality of a plurality of views, comprising a first view, 
a zoom view, a tilt view, and a pan view; a plurality of video clips, comprising a pan 
clip, a zoom clip, and a tilt clip, and an object clip; a plurality of viewpoints, each 
said viewpoint comprising a subset of said plurality of views, a subset of said 

* 

plurality of video clips, a viewpoint location and a viewpoint icon; a photo-realistic 
object, said object comprising said object clip, an object location and an object icon; a 
photo-realistic representation of physical space, said representation comprising said 
plurality of viewpoints and said object, wherein said object icon identifies said object 
location, and wherein said viewpoint icons identify said viewpoints locations for said 
viewpoints; a blueprint, wherein said blueprint is a two-dimensional cross-sectional 
view of said representation, wherein said blueprint includes said viewpoint icon and 
said object icon; and a compass, comprising a direction of a current view, wherein 
said compass indicates said current direction of said current view; and a navigation 
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♦ * 

subsystem, including- a zoom tool, wherein said zoom tool provides for the 
navigation from said first view to said zoom view using said zoom clip; a tilt tool, 
wherein said title tool provides for the navigation from said first view to said tilt 
view using said tilt clip; and a pan tool, wherein said pan tool provides for the 

navigation from said first view to said tilt view using said pan clip, as recited in 

* 

claim 35. 

♦ 

Conclusion 

.' 15. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Any inquiry concerning this communication or earlier communications from 

■ 

the examiner should be directed to X. L. Bautista whose telephone number is (571) 

272- 4132. The examiner can normally be reached on Tuesday-Friday 8-00AM- 
6:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiners supervisor, Weilun Lo can be reached on (571) 272*4847. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 

273- 8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair~direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 

- 

automated information system, call 800-786-9199 (IN USA OR CANADA) or 571- 

i 

272-1000. _ 
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